i  D-TR-6 8-11 
iTI  FILE  COPY 


ESD-TR-68-1I 


as 


Technical  Report  442 


MASSACHUSETTS  INSTITUTE  OF  TECHNOLOGY 


LINCOLN  LABORATORY 


TIME  OPTIMAL  CONTROL 

FOR  A  CLASS  OF  COMMON  RANDOM  DISTURBANCES 


N.  P.  SMITH 
Group  76 


TECHNICAL  REPORT  442 

2  FEBRUARY  1968 


LEXINGTON 


MASSACHUSETTS 


TIME  OPTIMAL  CONTROL 

FOR  A  CLASS  OF  COMMON  RANDOM  DISTURBANCES 


ABSTRACT 

This  report  concerns  the  time  optimal  control  of  a  system  variable  where  the  controlling 
input  to  the  system  is  bounded,  as  is  normally  the  case  in  practice.  Optimal  control  is 
defined  here  as  that  control  which  yields  time  optimal  trajectories .  It  is  shown  that  time 
optimal  control  also  yields  optimal  trajectories  in  the  sense  of  minimizing  the  maximum 
error  (if  this  is  the  initial  error,  minimize  the  overswing  next)  and  the  number  of  oscil¬ 
lations.  The  problem  of  optimal  control  of  a  second-order  system  initially  in  equilibrium 
and  subjected  to  a  large  class  of  commonly  occurring  random  disturbances  is  solved.  Dis¬ 
turbances  are  considered  to  be  controllable  or  uncontrollable.  The  broad  class  of  random 
disturbances  treated  herein  may  have  initial  nonequilibrium  values  and  consist  of  a  unidi¬ 
rectional  uncontrollable  portion,  followed  by  a  controllable  portion  of  sufficient  duration  to 
enable  an  optimal  controller  to  bring  the  system  to  equilibrium.  A  single  control  function 
is  derived  which  suffices  to  yield  optimal  trajectories. 

Previous  attempts  which  were  made  to  solve  similar  problems  using  a  statistical  approach 
succeeded  only  in  obtaining  approximate  optimal  control  for  disturbances  restricted  to  a 
specific  class  such  as  white  noise.  Brownian  motion,  etc.  In  this  report,  the  optimal  con¬ 
trol  applies  to  a  much  larger  class  of  random  disturbances  than  previous  results,  in  that 
it  does  not  restrict  the  disturbance  to  be  a  member  of  a  single  statistically  defined  class 
such  as  white  noise,  etc.  The  disturbances  treated  here  contain  the  entire  class  of  dis¬ 
turbances  most  commonly  occurring  in  practice.  This  is  true  because  in  modern  control 
design  the  maximum  magnitude  of  the  controlling  input  is  sized  such  that  most  of  the  time 
the  disturbances  are  considered  controllable.  Thus,  the  most  general  class  of  commonly 
occurring  disturbances  of  interest  to  the  control  designer  are  those  which  are  uncontrol¬ 
lable  for  a  short  duration,  followed  by  long  controllable  portions. 

The  success  of  the  technique  used  in  this  report  lies  in  assuming  that  the  disturbance  is 
initially  known  in  advance.  In  this  way,  a  single  control  function  is  derived  that  will  yield 
optimal  trajectories  for  as  large  a  class  of  disturbances  as  mathematically  possible.  This 
is  the  best  that  can  theoretically  be  achieved.  Terms  contained  in  the  derived  control  func¬ 
tion  yield  those  statistical  parameters  which  are  actually  required  to  obtain  optimal  control. 
Once  the  basic  form  of  the  necessary  statistical  parameters  has  been  obtained,  the  designer 
can  estimate  or  measure  what  these  parameters  are  in  the  actual  system  for  practical  op¬ 
timal  control.  Experimental  results  verifying  this  fact  are  presented  here. 
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TIME  OPTIMAL  CONTROL 

FOR  A  CLASS  OF  COMMON  RANDOM  DISTURBANCES 


I.  INTRODUCTION 

1 

In  1944,  Oldenburger  derived  the  time  optimal  response  of  a  seeond-order  system  to  step 
changes  in  the  load  and  reference  value  of  the  controlled  variable  while  trying  to  optimize  air¬ 
craft  propeller  governors.  This  system  was  subject  to  a  saturation  condition  on  the  manipulated 
(controlling)  variable.  In  the  speed  governor  field,  the  saturation  condition  is  always  satisfied. 
Optimal  response  is  obtained  by  having  the  servomotor  piston  speed  at  maximum  or  zero  at  all 
times . 

2 

A  paper  on  optimal  nonlinear  control  by  D.  MacDonald  appeared  in  1950,  with  subsequent 
papers  by  Hopkin,^  Bogner  and  Kazda,4  Bushaw,^  LaSalle,^  and  many  others.  Hopkin  and  Wang7 

g 

studied  relay  systems  for  random  inputs.  Rozonoer  published  a  series  of  papers  on  the 
Pontryagin  maximum  principle  concerned  with  minimizing  the  duration  of  trajectories  or  other 
functionals,  but  this  elass  does  not  include  the  maximum  error.  For  the  system  treated  in  this 
report,  it  is  shown  that  by  minimizing  error  response  time  the  maximum  error  is  simultane¬ 
ously  minimized. 

9 

In  1953,  Flugge-Lotz  published  a  book  describing  in  detail  the  design  and  performance  of 

discontinuous  control  systems  using  combinations  of  linear  control  functions.  A  paper  on  optimal 

10 

control  for  step  and  pulse  disturbances  was  written  in  196  5  by  Oldenburger  and  Chang,  who 

presented  the  optimal  control  functions  for  step  and  pulse  disturbances  of  arbitrary  magnitude 

1 1 

and  duration.  In  1956,  Bellman,  Glieksberg,  and  Gross  presented  a  general  solution  to  the 
problem  of  optimizing  error  response  time  of  a  system  from  any  given  initial  state  to  equilib¬ 
rium,  while  the  system  experieneed  no  external  disturbance  during  this  transition.  In  1957, 

12 

Pontryagin,  et  al.,  published  a  book  which  introduced  the  maximum  principle.  In  this  work, 

the  general  solution. for  optimizing  the  error  response  time  was  presented,  where  the  controlling 

1 3 

variables  were  subject  to  a  saturation  limitation.  In  196  5,  Woodside  described  an  experimental 
method  for  finding  approximate  solutions  to  problems  in  optimal  control,  and  also  obtained  ex¬ 
perimental  results  for  a  number  of  examples  using  this  method.  The  systems  were  subjected 

14 

only  to  a  smoothed  white  noise  input  and  a  Brow'nian  motion  input.  In  1966,  Oldenburger 

published  a  book  concerned  with  optimizing  entire  response  curves  rather  than  a  single  index 

of  performance.  Ilis  treatment  of  optimal  control  for  arbitrary  disturbances  involved  using  a 

discrete  approach  and  then  going  through  a  limiting  process  to  obtain  the  optimal  control  function. 
1  5 

In  1966,  Kushner  wrote  a  series  of  papers  concerned  with  the  existence  and  sufficient 
conditions  of  optimal  stochastic  control.  He  was  concerned  with  minimizing  the  expected  value 
of  a  functional.  Athans^  wrote  a  paper  on  optimal  control  with  bounded  variables,  and  was 
concerned  with  minimizing  energy,  fuel,  and  time. 
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In  this  report,  a  system  with  input  l  is  considered,  where  l  is  a  disturbance  to  which  the 
system  is  subjected;  for  convenience,  this  will  be  called  a  load  disturbance.  The  system  is 
taken  with  one  output  only,  this  being  the  controlled  variable  x.  In  addition  to  the  disturbance 
input,  there  is  another  quantity,  namely,  the  controlling  variable  m  which  is  to  be  varied  by 
the  controller  to  keep  x  constant  or  varying  with  time  according  to  a  reference  value  r;  i.e., 
it  is  desired  to  have  r  x  at  all  times.  Let  e  denote  the  error  (r  —  x).  In  the  normal  equilib¬ 
rium  state,  r  x  0. 

For  a  constant  r,  many  controlled  systems  may  be  represented  by  the  equation 

x'  =  K^m  -  Kz*  (1) 

where  and  are  constants.  Let  m*  denote  the  rate  of  change  dm/dt  of  m  with  respect  to 
time  t.  In  physical  problems,  m1  is  bounded  so  that 

|m'|  <  K3  (2) 


for  a  constant  and  the  absolute  value  |m'|  of  m'.  In  many  problems,  except  for  such  factors 
as  lags,  m'  can  be  made  at  any  instant  to  take  on  any  value  between  —  and  +Ky 
By  the  substitutions 


X  = 


K1K3 


m' 


VS 


(3) 


Relations  (1)  and  (2)  become 

X’  =  M  —  L  (4) 

|u|  1  .  (5) 

Differentiating,  Eq.  (4)  becomes 

X"  =  u  -  L*  .  (6) 

When 

I L*  |  <1  (7) 

the  load  disturbance  is  said  to  be  controllable.  The  equilibrium  state  is  defined  by 

X  =  XT  =  0  (8) 

Now,  if  the  system  starts  from  State  (8),  but 

|L'|  >  1  (9) 

it  follows  that  X"  ^  0,  and  a  system  error  X  with  X'  ^  0  arises.  A  perfect  controller  cannot 
prevent  the  error,  and  the  disturbance  is  uncontrollable.  If 

|L'|  =1  (10) 


in  practice,  an  error  will  always  arise,  and  one  cannot  make  (u  —  L' )  positive  or  negative  as 
desired  to  bring  the  system  back  to  the  equilibrium  state.  Thus,  the  Cases  (9)  and  (10)  are 
both  uncontrollable.  A  unidirectional  disturbance  is  defined  as  a  disturbance  for  which  the  sign 
of  L*  is  constant.  Figure  1  is  a  typical  plot  of  L*  where  controllable  and  uncontrollable  portions 
of  the  L*  vs  t-curvc  alternate.  The  controllable  portions  are  indicated  by  C,  and  the  uncon¬ 
trollable  portions  by  U.  In  practice,  violent  increases  in  L  followed  by  violent  decreases  will 


2 


usually  be  rare;  also,  is  chosen  so  that,  for  normal  disturbances,  the  uncontrollable  por¬ 
tions  are  followed  by  long  controllable  sections  (see  Ref.  14,  pp.  193  and  194). 
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Fig.  1.  L'  vs  time,  illustrating  typicol  controlloble  ond  uncontrol loble  intervols. 


Suppose  that  the  disturbance  L(t)  is  known  not  only  for  the  past  and  present,  but  also  for 


all  future  time.  It  is  then  possible  to  derive  the  optimal  control  function  for  this  disturbance. 


This  control  function  can  then  be  utilized  for  the  ease  when  the  future  of  the  disturbance  is  not 
known.  In  this  way,  an  optimal  control  function  can  be  obtained  for  as  large  a  elass  of  disturb¬ 
ances  as  mathematically  possible.  Since  no  control  ean  be  better  than  that  obtained  when  the 
disturbance  is  known  in  advance,  a  bound  is  set  on  the  actual  response  which  ean  be  attained. 

Optimal  control  shall  be  referred  to  as  that  control  which  yields  the  minimum  duration  of 
the  error  response  when  the  future  of  the  disturbance  L(t)  is  known  arbitrarily  far  in  advance. 
This  report  treats  the  problem  of  obtaining  the  control  function  which  yields  optimal  control  in 
the  sense  of  minimizing  the  duration  of  the  error  response.  It  is  also  shown  that  this  control 
simultaneously  minimizes  the  maximum  absolute  error  |x^|  in  the  controlled  variable  X  (if 
this  is  the  initial  error,  minimize  the  first  overswing  next)  and  the  number  of  oscillations. 

This  report  proves  that  a  unique  optimal  trajectory  exists  for  the  system  of  Eq.  (6),  with  L(t) 
consisting  of  a  random  unidirectional  uncontrollable  portion  followed  by  a  random  controllable 
portion  of  sufficient  duration  to  enable  the  optimal  controller  to  bring  the  system  to  equilibrium. 
It  is  also  shown  that  this  optimal  trajectory  is  obtained  by  having  bang-bang  control;  i.  e.,  the 
controlling  variable  u  takes  on  its  saturation  value 


(11) 


until  the  system  reaches  the  equilibrium  state.  The  optimal  control  function  2  is  defined  as  a 
function  which  causes  u  to  take  on  its  optimal  value  (for  optimal  control)  at  all  times.  It  is 
shown  that  optimal  control  is  obtained  by  letting 


u 


-sgn  2 


(12) 


where 


u  -  —  1 


2  >  0 


u  =  +1 


2  <  0 


u  ~  0 


(13) 
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A  phase  is  defined  as  a  portion  of  the  solution  for  which  u  is  a  constant.  In  this  report,  an 
optimal  control  function  2  is  derived  which  depends  on  the  instantaneous  values  of  the  variables 
X,  X',  L,  and  on  the  average  values  of  L  and  L'  over  a  future  duration  of  time.  It  is  also  shown 
that  the  single  control  Law  (12)  suffices  to  yield  optimal  trajectories.  Figure  2  is  a  block  dia¬ 
gram  of  the  optimal  system. 


Fig.  2.  Block  diogrom  of  optimol  control  system. 


II.  THE  MAXIMUM  PRINCIPLE 

The  purpose  of  this  section  is  to  use  the  maximum  principle  to  show  the  existence  of  time 
optimal  trajectories  to  equilibrium,  for  a  system  initially  at  equilibrium  and  subjected  to  a 
unidirectional  uncontrollable  disturbance  followed  by  a  controllable  portion  of  sufficient  duration 
to  enable  an  optimal  controller  to  bring  the  system  to  equilibrium.  The  disturbance  may  or  may 
not  have  a  step  at  t  -  0.  It  will  be  shown  here  for  what  general  conditions  optimal  solutions  exist, 
and  that  optimal  control  is  obtained  with  a  unique  two-  or  one-phase  trajectory  to  equilibrium. 

The  system  described  by  Eqs.  (5)  and  (6)  may  be  put  in  the  symbolism  normally  employed 
for  applying  the  maximum  principle.  The  variables  x^,  x^,  x^,  and  u  shall  be  used  where 

xJ[  ~  X  ,  x^  =  X1  ,  u  =  M’  ,  x3  =  t  .  (14) 

The  system  of  Eq.  (6)  becomes 

xi  “  x2 

x!>  =  u  -  L* 


1  u  I  «  1  .  (15) 

Subscript  o  denotes  variables  in  the  initial  state;  state  (x^q,  x^q)  *s  defined  as  the  state  of  the 
system  at  t  =  t  so  that  (x^,  x^)  (x1Q,  x2q^  at  t  tn'  The  initial  conditions  at  time  t  tQ  are 


X1(V  =  xio 


X2<V  :  X20 


x3(to 


)  =  C 


L(t  )  =  L 
v  o  o 


(16) 


Times  t1  and  t^  are  the  terminal  times  of  the  first  and  second  phases,  respectively,  of  a 
time  optimal  two-  or  one-phase  trajectory  to  equilibrium.  States  (x^i>x2i^  anc*  ( xi2 '  XZZ^  are 
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the  states  of  the  system  at  times  t  t^  and  t  t^,  respectively,  so  that  (x^,x^)  (x^x^) 

at  t  t1#  and  (x^,x^)  =  (x12>x22)  at  t  t2’  At  the  terminal  time  t  t^,  it  is  required  that  the 
system  be  at  the  equilibrium  state 

xl(t2)  X12  0 


X2^2*  x22  ° 

Pontryagin's  function  H  is  given  on  p.  60  of  Ref.  12  by 

H  *  f  +  *.f.  +  ¥, f,  +  +  _f_ 
oo  11  22  33 


where 


(17) 


(18) 


x*  f 
o  o 


X'l  fl  X2 


X2  =  f2  (U"L,) 


x'  f  1 
3  3 


(19) 


When  the  disturbance  L(t)  is  known  arbitrarily  far  in  advance,  it  may  be  expressed  in  terms  of 

t  only.  According  to  p.  59  of  Ref.  12,  the  auxiliary  unknown  x^  defined  by  x^  t  is  used  in  place 

of  t  so  that  Eqs.  (19)  may  be  written  in  a  form  not  depending  explicitly  on  t.  For  minimum 
1 2 

time,  Pontryagin  gives  f  1.  Now,  Relation  (18)  becomes 

H  *  +  x2^1  +  (u  -  L' )  .  (20) 

In  Ref.  12,  it  is  shown  that 

'I'  -  constants  0  .  (21) 

o 

Thus,  is  a  nonpositive  constant.  If  ^  0,  it  follows  that  H  becomes  a  maximum  with 
respect  to  u  when 

u  sgn*2  .  (22) 


ri  he  adjoint  system  is  given  by 


*  ’  0 
1 


*3  *2L’ 


The  solution  of  Eqs.  (2  3)  yields 


*i  ki 


*2  k2"klt 


4'  3  -  ktL  +  L,(k2-k1t)  +  k3 


(23) 


(24) 
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for  constants  of  integration  k^,  k^,  and  k^.  From  Ref.  12,  at  the  terminal  time  t  t^,  the 
condition 

*3(t2)  0  (25) 


must  hold  to  satisfy  the  transvcrsality  condition.  By  Relations  (20)  and  (24),  H  becomes 

H  =  'b  o  +  kj(x2  +  L)  +  k^  +  u(k2  —  k^)  .  (26) 

By  the  second  of  Relations  (24)  and  Eq.  (22),  if  (k^,  k^)  (0,  0)  it  follows  that  u  switches  once 

at  most  from  the  u  =  +1  to  u  =  —  1,  or  vice-versa. 

Suppose  the  system  is  on  a  one-  or  two-phase  trajectory  to  equilibrium  with  t  0.  Con¬ 
sider  all  two-  or  one-phase  trajectories  for  which  u  =  —  1  for  the  first  phase  until  t  t^,  and 
then  u  +1  for  the  phase  leading  to  equilibrium  at  t  t^.  For  the  single -phase  case,  0. 

Consider  first  the  two-phase  case,  from  which  the  duration  of  each  phase  must  be  positive,  i.e., 
t^  >  0,  and  (t2  —  t^)  >  0.  Later,  necessary  and  sufficient  conditions  are  derived  which  insure  the 
existence  and  positive  duration  of  both  phases.  These  are  restrictive  conditions  on  the  initial 
state  of  the  system  and  on  the  disturbance  L(t). 

The  response  of  the  system  of  Relations  (15)  along  the  first  phase  u  =  —  1  of  a  two-phase 
trajectory  is  obtained  by  direct  integration  as 


X1  =  X10  +  X20l  ~  V  (1  +  ^  •  t>0 


where  the  quantity  tL  is  defined  by 
tL  =  §  L  dt 


(27) 


(28) 


Letting  u  =  —  1  and  eliminating  x^  from  Eqs.  (26)  and  (27)  yields  II  =  max  H  along  the  first  phase, 
where 


maxH  =  +  k.x_  —  k_  +  k-.  .  (29) 

o  1  20  2  3 

Since  t^  is  the  duration  of  the  first  phase,  the  response  x^ft)  along  the  second  phase  is  obtained 
from  the  second  of  Relations  (15)  and  (28)  as 

X2  =  x20  ~~  2tl  +  *  ”  L  »  t  .  (30) 

Along  the  second  phase  u  =  +1,  Eqs.  (26)  and  (30)  yield 

max  H  =  ^+k.(x—  2t,)  +  k_  +  k.  .  (31) 

O  1  20  1  2  3 

Setting  t  =  t2  in  Eqs.  (30)  with  x^t^)  0  yields 

(t2_tl)  =  (tl  _X20  +  L2>  •  <32> 

Since  the  duration  of  the  second  phase  must  be  positive,  it  follows  from  Eq.  (32)  that  a  neeessary 
condition  for  the  existence  of  two-phase  optimal  trajectories  to  equilibrium  be 

(tj  -  x20  +  L2)  >  0  ,  ti>0  (33) 


6 


where  L^  =  L(t2).  Later,  it  will  be  shown  that  Relation  (33)  is  always  satisfied  for  the  disturb¬ 
ances  and  initial  states  treated  in  this  report.  Setting  max  H  =  0  in  Eqs.  (26)  and  (29),  and 
eliminating  t^  from  Eqs.  (25)  and  (32),  yield  three  linear  homogeneous  equations  in  k^  k^,  and 


0  =  *  +  k  x  -  k  +  k 

o  1  20  2  3 


>I'o+kl(X20-2tl)  +  k2  +  k3 


0  k1[L2(l-I^)  +  L'(x20-2t1)l  +  L'k2+k3 


(34) 


where  =  L1 (t^)-  By  Relation  (21),  the  quantity  'k q  is  a  negative  constant,  or  zero.  Consider 
first  the  case  where  is  a  negative  constant.  Because  one  of  the  'I's  is  redundant,  we  may 
take 

*  =  -1  .  (35) 

o 

The  solutions  of  Eqs.  (34)  and  (35)  for  k^,  k^,  and  k^  are 
kl  (1-L')  <*20 

k _ 5 _ 

2  (l-I^)(x20-t1-L2) 

~L2  -L2 

k3  '  (1  -  Lp  (x2Q  -  t3  -  L2)  +  (1  -  1^)  •  (36) 

The  disturbance  L(t)  is  assumed  to  be  controllable  during  the  second  phase,  hence, 

|L'  |  <  1  ,  t4  <  t  <  t2  .  (37) 

It  follows  from  Relations  (32),  (36),  and  (37)  that  if  (t2  —  t^)  >0,  and  t.  >  0  are  satisified,  then 
the  conditions 

kl  <  0  '  k2<  0  (38) 


hold.  By  Relation  (21),  other  possible  optimal  trajectories  may  exist  for  the  case  where 

*  0  (39) 

o 

instead  of  ^  =  —1.  There  is  a  solution  of  max  H  -  0  for  the  k's  not  all  vanishing,  and  hence  of 

the  'k's  not  all  vanishing  identically,  only  if  the  determinant  of  the  coefficients  in  Eqs.  (34)  is 
zero.  This  condition  is  given  by  the  relation 

(1  “  L2}  (x20  “  L2  "  V  “  0  •  <40> 


By  Relations  (32)  and  (37),  it  follows  that  Condition  (40)  is  a  necessary  restrictive  condition  on 
x ^ q  and  L(t^)  for  the  existence  of  optimal  single-phase  trajectories  (u  =  — 1)  to  equilibrium. 
Later,  it  is  shown  that  Condition  (40)  does  not  occur  for  the  initial  states  and  disturbances 
treated  here;  thus,  it  may  be  disregarded  for  the  optimal  control  of  this  report. 


7 


By  Relations  (24)  and  (38),  it  follows  that 

0  <  t  <  tA 

t  =  ti 

.  (41) 

It  has  therefore  been  shown  that 

*2(t)  ^  0  ,  t  £  ti  (42) 

for  the  case  of  two-phase  trajectories  to  equilibrium,  with  u  =  — 1  followed  by  u  =  +1.  Consider 
trajectories  with  a  single  phase  (u  =  4-1 )  to  equilibrium.  For  this  case,  let  t^  0.  In  the  limit, 
it  follows  from  Relations  (24)  and  (36)  that 

lim  'I'  -  —  k  t  .  (43) 

tro 

Thus,  by  Relations  (38)  and  (43),  the  relations 
*2  0  t  0 

*2  >  0  ,  t  >  0  (44) 

hold  for  the  case  of  a  single  phase  to  equilibrium.  The  theory  for  the  symmetrical  case  of  one 
or  two  phases  to  equilibrium,  with  u  =  —  1  during  the  phase  leading  to  equilibrium,  is  obtained 
in  the  same  manner  as  in  the  preceding  argument. 

By  Relations  (22)  and  (41),  H  is  a  maximum  with  respect  to  u  when  u  takes  on  the  values 

u  -  —  1  ,  0  ^  t  <  t . 

1 

u  =  4-1  ,  t!  <  t  <  t  .  (45) 

It  has  also  been  shown  by  Relation  (36)  that  there  are  values  of  k^,  k2,  and  k^  such  that 
max  H  =  0  for  4>2  £  0  over  the  duration  of  the  trajectory.  It  follows  from  the  maximum  principle 
that  the  control  Schedule  (45)  is  a  necessary  condition  for  time  optimal  trajectories  to  equilib¬ 
rium.  To  show  the  existence  of  optimal  trajectories  requires  only  the  proof  that  the  Schedule  (45) 
actually  transfers  the  system  from  the  initial  state  (x^Q,  x20)  to  terminal  state  (0,  0)  while 
being  subjected  to  the  disturbance  L(t). 


*2<0 
2  0 

*2>0 


HI.  EXISTENCE  OF  TIME  OPTIMAL  TRAJECTORIES 

In  this  section,  it  is  shown  that  unique,  two-phase  time  optimal  trajectories  to  equilibrium 
exist  for  the  case  of  a  unidirectional  uncontrollable  disturbance  followed  by  a  controllable  por¬ 
tion  of  sufficient  duration  to  enable  an  optimal  controller  to  bring  the  system  to  equilibrium.  It 
is  also  shown  that  these  time  optimal  trajectories  are  also  optimal  in  other  engineering  senses 
such  as  that  the  maximum  error  is  minimized  and  there  is  no  overshoot. 

Consider  the  system  of  Eqs.  (15)  with  the  initial  states 


x 


20 


=  — L  ^  0 
o 


(46) 
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and  disturbance  L*  consisting  of  a  unidirectional  uncontrollable  portion  which  terminates  at 
t  t  .  For  t  >  t  ,  the  disturbance  L*  is  controllable  until  the  system  reaches  the  equilibrium 
state  at  t  t^.  We  have  thus  defined  a  class  of  disturbances  L*  satisfying  the  conditions 

,  o  <  t  <  tu 

|L*'|<1  ,  tu<t<t2  .  (47) 


First,  we  treat  the  case  where 

L*  1  <  — 1  (48) 

for  the  uncontrollable  interval  0  <  t  <  t  ,  as  shown  in  Fig.  3.  In  order  for  the  system  to  reach 
equilibrium  along  a  time  optimal  two-phase  trajectory,  we  must  have  u  —1  for  the  first  phase. 
Hence,  the  -curve  will  bend  away  from  the  t-axis  a  minimum.  This  control  will  result  in  a 
minimum  error  for  each  instant  t  in  the  interval  (0,  t  ).  From  Eqs.  (15)  and  the  fact  that 
L*'  <  — 1  during  the  interval  0  <  t  ^  t  ,  it  follows  that 

x  >  0 
1 

x  ^  >  0  ,  0  <  t  ^  t 

2  u 

as  shown  in  Figs.  3  and  4.  For  t  >  t  ,  the  disturbance  L*  becomes  controllable.  It  follows  by 

Hqs.  (15)  that  if  u  =  — 1  holds  for  t  >  t  ,  at  some  time  t  t..  where  tn  >  t  ,  we  have 
n  u  M  M  u 

Xl(tM)>  0 


X2(tM)  "  ° 


(49) 


Fig.  3.  System  response  to  uncontrol loble  disturbonce 
followed  by  control  loble  disturbonce. 


Fig.  4.  System  trojectories  for  various 
switching  times. 
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and  x1  reaches  the  maximum  value  of  x^  at  time  t  t^.  It  follows  from  Eqs.  (15)  that 


0  <  t  <  t 


M 


(50) 


as  shown  in  Figs.  3  and  4. 

If  u  =  — 1  holds  for  t  >  t^,  the  inequality  lL*  '|  <  1  implies  that  x^  <  0.  We  assume  that 

there  is  a  number  p  and  an  instant  t„  such  that  for  t  >  t„,  where  tA  >  t  ,  we  have 

4  4'  4  u 


|l*  '|  <  n  <  i 


(51) 


It  follows  that  with  u  =  —  1,  the  x^ -curve  will  eventually  reach  the  t-axis.  Let  t  denote  the  time 
at  which  the  x^-curve  crosses  the  t-axis,  as  shown  in  Fig.  4.  For  any  vertical  line  £  between 
t^j  and  t(  ,  the  area  between  the  x^ -curve,  the  t-axis,  the  line  t  t^.,  and  the  line  £  is  a  mini¬ 
mum.  At  the  crossing  point  t  t  f  we  have 


(52) 


The  eurve  from  t  =  t^  to  t  t  is  not  optimal  in  the  sense  that  this  results  in  an  overswing 

for  t  >  t  with  x,  xJ  at  the  minimum  point  for  x,  <0.  To  minimize  — x,  ,  we  must  have 
c  1  lm  1m  lm 

u  -  + 1  for  t  >  t  .  The  value  of  —  x|m  ean  be  diminished  further  by  letting  u  switch  from  —  1  to 

+  1  at  a  point  and  time  t  =  t^,  where  t^  <  t  .  As  P^  moves  to  the  left  in  Fig.  3,  the  minimum 

point  xlm  rises.  If  we  take  P^  at  the  maximum  point  t  t  the  x^-curve  will  rise  to  the  right 

of  t„T  as  shown  in  Figs.  3  and  4.  It  follows  that  there  is  a  time  t  t.  between  t.,  and  t  such  that 
M  B  1  M  c 

t  <  t  <  t  (53) 

M  1  e  1  ; 

where  the  x^ -curve  to  the  right  of  P^  touches  the  t-axis  for  a  value  t  -  t^.  This  point  is  the 
equilibrium  state 


xl(t2)  X2(t2>  =  ° 


Starting  at  t  =  t^,  the  arc  for  which  u  +1  up  to  the  equilibrium  state  at  t  t  is  denoted  by  T' 
in  Figs.  3  and  4.  The  curve  in  Fig.  3,  composed  of  the  x^ -curve  from  t  0  to  t  t^  (for  which 
u  -  —  1)  and  the  T1 -curve  from  t-t^tot  t?  (for  whieh  u  -  +1),  is  eallcd  the  T-eurve  and  is 
optimal,  as  shown  by  the  following  argument. 

Every  other  solution  T^  passes  above  or  coincides  with  the  T -curve  from  t  0  to  t  t^,  or 

is  identical  with  it  up  to  a  value  t^  of  t,  where  t^  <  <  t^,  after  whieh  it  erosses  the  t-axis  and 

attains  a  minimum  point  below  the  t-axis  at  an  instant  t  t  where  t  >  t_,,  as  shown  in  Fig.  3. 

^  mm2  s 

If  T1  T  for  0  <  t  t^,  then  for  t  >  t^  any  curve  T^  will  lie  on  or  between  the  curve  u  —1  and 
TL  In  fact,  for  any  t  where  t  >  t^ 


-(1  +  L*')4x'24  (1  -  L*  *) 


The  slope  x^  of  the  curve  with  u  =  — 1  decreases  at  the  maximum  rate,  whereas,  for  T  to  the 
right  of  t  =  t^,  it  increases  at  the  maximum  rate. 
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The  T-eurve  is  unique  and  optimal  in  that  all  other  solutions  yield  overswings  and  greater 
durations  from  t^  to  equilibrium.  We  have  shown  that  the  maximum  error  x^  is  minimized  by 
a  unique  two-phase  trajectory  to  equilibrium  with  a  phase  u  =  — 1  to  t  =  t^  followed  by  a  phase 
u  =  +1  to  t  t2,  where  (x^*^)  =  ( 0,  0)  at  t  =  t^.  It  follows  that  the  conditions 


x2  <  0  ,  tM<t<t2  (54) 

also  hold  along  the  optimal  trajectory.  A  similar  argument  holds  for  the  ease  where  x^Q  0, 

x™  ^  0,  and  L*  1  >  +1  for  the  uncontrollable  interval  0  <  t  ^  t  ,  where  t  <  t . . 

20  u  u  1 

By  the  preceding  argument,  we  have  proved  the  existence  of  unique  two-phase  trajectories 
to  equilibrium  with  t^  >  0,  and  (t  —  t^)  >  0  for  the  system  of  Eqs.  (15)  subjected  to  the  elass  of 
disturbance  L*  .  It  follows  that  a  unique  time  optimal  solution  exists  for  whieh  Relations  (38) 
and  (41)  hold  and  the  control  Schedule  (45)  yields  max  H  =  0  with  ^  ^  0  along  the  T-eurve. 

IV.  OPTIMAL  RESPONSE  OF  SYSTEM 


Equations  are  now  derived  which  describe  the  optimal  response  of  the  system  of  Eqs.  (15) 
when  subjected  to  a  disturbance  L(t).  General  restrictive  conditions  on  L(t)  are  given  whieh 
must  be  satisfied  in  order  for  the  system  to  reach  equilibrium  along  an  optimal  two-  or  one- 
phase  trajectory.  It  is  verified  here  that  the  elass  of  disturbances  L*  treated  in  this  report  do 
satisfy  these  conditions. 

Let  the  quantities  LL  and  cv„  be  defined  by 


Lij  (t 


■t. 

J  L'  dt 


a . .  - 

ij 


(t 


—  f 

j  -  V  \ 

—  - 2  \  tj  <*i  -  L' 

—  -  t.)  Jt-  J 


i  =  0,  1;  j  =  1,  2 


dt 


(55) 


By  direet  integration  of  Eqs.  (15)  with  u  =  —  1  over  the  interval  from  t.  0  to  t.  t.,  the  state 


of  the  system  at  t  t^  beeomes 


X21  X20  tl*1  +  L01) 


X11  X10  +  X20tl  2  *1  ^  +  ®0l* 


(56) 


where  the  initial  value  of  x  ^  arises  from  an  initial  value  of  L  at  t  -  0, 


x~~  —  L 

20  o 


(57) 


Let  (x^,  x  ^)  be  ^e  initial  state  of  the  system  at  the  start  of  the  second  phase.  The  state  of  the 
system  at  the  terminal  time  t  =  t^  is  also  obtained  by  direet  integration  of  Eqs.  (15)  with  u  =  +1 


over  the  interval  from  t.  =  t.  to  t ..  =  t0,  thus, 

i  1  j  2 


X22  =  X21  +  (t2“tl)  (1  -L12> 


X1 2  X11  +  ^2  V  X21  +  2  ^2  V  **  °'l2) 


(58) 
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In  order  for  the  system  to  reach  equilibrium  at  t  -  along  a  phase  u  =  41  from  the  state 
( 1 '  X1 1 ) '  ^  f°ll°ws  from  Eqs.(17)  and  (58)  that 


'll 


J  (t2  -  t^2  (1  -  2L 


12 


a12^ 


(59) 


We  consider  only  disturbances  which  are  controllable  along  the  second  phase;  then,  by 
Relation  (37),  it  follows  that 

|  L(t)  -  L(t1  )|  <  (t  -  tt)  ,  t4<t<t2  .  (60) 


Hence,  it  follows  from  Relation  ( 6 0 )  and  the  first  of  Relations  (55)  that 


"121 


<  1 


(61) 


In  view  of  Relations  (55)  and  (37),  we  may  also  write 

- - - 5  \  2  |Mt)-L(t,)|  dt 

“Z-*!*  ‘l 


(«12-2Li2»< 


By  Relation  (37),  it  also  follows  that 

|L(t)  -  L(tz)|  <(t2-t)  ,  t4<t<t2 


(62) 


(63) 


Thus,  by  Relations  (62)  and  (63),  we  have 

I«12-2L'12I<1  •  (64) 

Since  (t^  —  t^)  >  0,  it  follows  from  Relations  (59),  (61),  and  (64)  that  the  state  of  the  system  at 
t  =  t^  must  satisfy  the  conditions 


*21  <0 


X11  >  0 


(65) 


By  Relations  (56),  we  may  express  Conditions  (65)  in  the  form 

x20-ti<1  +  i:bi,<0 


xi  o  +  “zih  +  7  ‘l2*1  +  2Loi  —  a  oi >  *  0 


By  Relations  (46)  and  since  t.  >  0,  it  follows  that  the  conditions 


(1  +  L.'oi)  >  0 


(l  +  2L'i-50i)>0 


(66) 


(67) 


must  be  satisfied  in  order  for  Conditions  (65)  to  be  satisfied.  We  shall  now  show  that  Conditions 
(67)  are  satisfied  for  the  class  of  disturbances  L*  treated  in  this  report. 

By  Eqs.  (27)  and  (49),  the  state  of  the  system  at  t  =  t^  is 


12 


(68) 


X2(tIVp  '  °  X20  t]VI<  1  +  LOIVI) 


Xl(tM^  X20lM  2  (1  +  “OM*  lM 


where 


D  =lf 
oM  HiK 

_2_  rl 

01 OM  ~  2  \ 


L*  1  dt 


%  -  «  L *  ’  dt 


(69) 


M 


Eliminating  x^0  from  Relations  (68)  yields 


X1M  2  lM  (1  +  2L0M  “OM* 


(70) 


Since  x^  >  0  and  x>Q  >  0,  it  follows  from  the  first  of  Relations  (68)  and  Relation  (70)  that 


(1  +  L0M>>0 

<1  +  2Lom-%m)>0 


We  define  the  quantities  and  to  be 


(71) 


"Ml  =(t4-t)Jt  L*'  dt 

r  <>.  - 


t)  L*'  dt 


(72) 


M 


By  Relations  (55),  (69),  and  (72),  we  have 
t 


T  '  M  T  »  + 

J01  t,  OM 

1 


(i  _  hA  l' 

V  )  lmi 


«01  2  ^M^OM  Z,J0M)  +  aMl(tl  tM)  +  2L0MtlVItl' 


(73) 


Since  L*  is  controllable  over  the  interval  t.^  ^  t  ^  t^,  we  have 


Hu'  <4 


<  1 

'  Ml 1 


(74) 


Hence,  by  Relations  (71),  (73),  and  (74),  it  follows  that  Conditions  (67)  are  satisfied  for  the 
class  of  disturbances  L*  and  initial  conditions  of  Relations  (46)  treated  in  this  report.  A  sim¬ 
ilar  argument  holds  for  the  case  where  =  0,  x^q  ^  0,  and  L* 1  >  +1  for  the  uncontrollable 
interval  t  4  t  . 


V.  OPTIMAL  CONTROL  FUNCTION 

We  now  introduce  a  control  function  2(t,  T)  and  prove  that  time  optimal  control  for  the  class 
of  disturbances  L*  and  initial  states  of  Relations  (46)  is  obtained  by  the  control  Law  (12). 
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Let  T  be  the  duration  of  the  second  phase  u  =  +1,  where 


T  =  <t2-tl) 


Let  L’  and  L  be  defined  by  the  relations 


i  rt+T 

L'  =TJt  L*'dt 


L  = 


r 


L*  dt 


(75) 


We  introduce  the  function  2(t,  T)  where 

2(t,  T)  =  Xjll  -  L'  sgnxj)"  +  J  x2  \xz\  '  1  +  2(L'  +  —  ~  Ij  )  sgnx2  .  (76) 

Eliminating  (t^  —  t^)  from  Relations  (59)  yields  2^  =  0,  where 

2(t1#  T)  .  (77) 


Note  that  Eq.  (76)  also  yields  2^-0  for  a  pre -equilibrium  phase  u  =  —  1  as  well  as  for  u  =  +1, 
where,  again,  t  =  t|  at  the  start  and  t  =  t2  at  the  end  of  this  phase. 

We  shall  now  prove  that  time  optimal  control  is  obtained  by  the  control  law 

u  =  — sgn  2(t,  T )  (78) 

where 


u  =  —  1 

Z(t,  T)  >  0 

u  =  +1  , 

2  (t,  T)  <  0 

u  =  0 

2(t,  T)  -  0 

(79) 


By  the  maximum  principle,  we  have  shown  that  time  optimal  control  is  obtained  by  the  control 
Schedule  (45).  From  Relations  (46)  and  Eq.  (76),  it  follows  that 

2(0,  T)  ^  0  .  (80) 


First,  consider  the  case  where  2(0,  T)  0.  According  to  Relations  (76),  this  can  happen  only 
if  the  relation 


1  +  2(Lf  + 


L  -  L 


)  sgn  x2 


=  0 


(81) 


is  satisfied  at  t  -  0.  If  Relation  (81)  is  satisfied,  we  have  u  =  0  at  t  =  0  by  Relations  (79).  It 
follows  by  Relation  (48)  and  the  second  of  Relations  (15)  that 

x2>0 


x2  >  0  ,  t  =  id 

xj[>  0  (82) 

where  6  is  a  small  positive  quantity.  Thus,  from  Relations  (76)  and  (82),  we  have 

2(6,  T)  >  0  .  (83) 
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By  Relations  (50),  (76),  and  (83),  it  follows  that 

2(t,  T)>0  ,  t<tM  (84) 

Thus,  the  eontrol  Law  (78)  yields  optimal  control  for  the  interval  0  <  t  ^  t^.  The  proof  of  the 
relation 

£(t,T)  >0  ,  tM  <  t  <t1  (85) 

follows  from  p.  212  of  Ref.  14.  For  a  given  t,  the  functions  E(t,T)  belong  to  a  one-parameter 
class  with  T  as  the  parameter.  We  have  shown  that,  until  t  t^  on  an  optimal  trajectory,  any 
member  2(t,  T)  of  this  elass  will  do  in  Relation  (78);  in  fact,  any  member  will  suffice  until  we 
reach  a  pair  of  values  (t^,  T^)  of  (t,  T)  for  which 

=  0  (86) 


holds.  The  function  2(t,  T)  will  ehange  sign  at  a  time  t  =  where  t^  <  t^  <  t  ,  as  shown  from 
the  following  argument.  Note  in  Fig.  3  that  if  we  remain  on  the  u  -  —  1  curve,  at  some  time  t  t 
we  have  x^(t^)  -  0,  and  x^t  )  <  0;  thus,  2(tp,  T)  <  0  for  any  T.  It  follows  that  for  any  T  there 
is  a  point  between  t  =  t^  and  t  =  t  where  the  function  2(t,  T)  changes  sign. 

By  Relations  (54),  we  have 

x d  >  0  ,  t  <  t  <  t  .  (87) 


We  have  shown  that  a  unique  two-phase  time  optimal  trajectory  to  equilibrium  is  obtained  with 

u  =  — 1  along  the  first  phase  until  2(t,T)  =  0,  followed  by  u  =  +1  along  the  second  phase  until 

t  -  t,.  It  follows  that  the  switch  at  t  =  t^  occurs  when  S(t^,T)  =  0.  Thus,  the  T-curve  in 

Fig.  3  is  optimal  where  t^  is  the  smallest  value  of  t  for  whieh  there  exists  a  value  of  t^  sueh 

that  2(t,T)  0  at  t  “  t^,  and  (x^x  j  -  (0,0)  at  t  =  t^.  A  similar  argument  holds  for  the  case 

where  x,rt  0,  x_,~  ^  0,  and  L*  *  ^  1  for  the  uncontrollable  interval  0  <  t  ^  t  ,  where  t  <  t,. 

10  20  u  u  1 

We  have  proved  the  following  theorem. 

Theorem  1. 


Let  a  seeond-order  system  [Eqs.  (15)]  be  at  equilibrium  for  t  ^  0,  and 
let  the  system  be  subjected  to  a  disturbance  L(t)  for  t  >  0,  where  L(t) 
is  made  up  of  an  initial  uncontrollable  portion  followed  by  a  controllable 
portion  sufficiently  long  for  an  optimal  controller  to  bring  the  system  to 
equilibrium.  Let  L  and  L'  be  the  averages 


T 


L  dt 


L'  dt 


(88) 


of  L  and  L*  for  the  time  interval  (t,  t  +  T),  and  let  the  eontrol  function 
2(t,  T)  be  defined  by  Eq.  (76).  Optimal  response  in  the  sense  of  minimum 
time,  maximum  error  minimized,  and  no  overshoot  is  obtained  with  the 
eontrol  law 


u  =  — sgn  2 (t,  T)  [Law  (78)] 

until  a  pair  of  values  (t^T^)  of  (t,  T)  are  attained  for  whieh 
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(89) 


l!(tlfT1)  =  0  . 

At  the  instant  t  t^,  control  is  switched  to  the  equation 
u  sgn 

and  remains  at  this  value  of  u  until  equilibrium  is  attained. 

A  similar  argument  holds  for  the  initial  states  =  0,  x^Q  <  0,  with  IV  >  +1  for  the  uncon¬ 
trollable  portion,  or  x^Q  -  0,  x^Q  <  0,  with  L'  <— 1  for  the  uncontrollable  portion.  Thus,  the 
results  of  Theorem  1  also  apply  for  these  conditions. 

VI.  DELAY  IN  SWITCHING 

In  this  section,  we  consider  the  effect  of  a  time  delay  in  switching  u  from  +1  to  —1,  or 
vice-versa,  for  the  optimal  control  of  this  report.  It  is  shown  that,  for  a  time  delay  in  switch¬ 
ing  u,  the  control  Law  (78)  with  the  control  function  of  Eq.  (76)  yields  near-optimal  trajectories 
to  equilibrium.  As  the  delay  in  switching  becomes  small,  the  near-optimal  approaches  the  op¬ 
timal  trajectory.  It  is  also  shown  here  that  a  single  control  function  of  Eq.  (76)  suffices  to  yield 
optimal  trajectories  to  equilibrium. 

1 

The  original  idea  upon  which  the  following  argument  is  based  is  credited  to  Oldenburger. 
Consider  the  case  where  u  -1  for  the  first  phase  of  an  optimal  trajectory  until  for  values  t^ 
of  t  and  T^  of  T  we  have  ^(t^T^)  0.  At  this  point,  u  switches  from  —1  to  +1  for  optimal 

control.  In  practice,  there  will  be  a  delay  in  switching.  Suppose  that  we  switch  at  t  -  (t^  4  c), 

where  c  >  0,  instead  of  at  t  t^  for  which  control  Law  (78)  holds.  Thus,  at  t  t^  we  leave 
the  optimal  trajectory  where  u  4-1  after  t  t^,  and  remain  on  u  —1  until  t  (t^  4  c).  This 

trajectory  is  non-optimal.  Since  u  =  —  1  on  the  non-optimal  trajectory  for  each  t  in  the  interval 

( t ^ ,  t^  +  e),  it  follows  that  at  each  instant  t  in  this  interval  both  x^  and  x^  are  less  than  they 
would  be  if  we  had  stayed  on  the  optimal  trajectory  (u  +1).  If  e  is  small  enough,  the  quantities 

(sgn  x^),  L,  L,  and  L'  occurring  in  2(t,  T^  —  t  4  t^)  are  the  same  at  t  ( t ^  +  c)  for  both  the 

optimal  and  non-optimal  trajectories.  Since  x^  and  x^  are  less  for  the  non-optimal  than  for  the 
optimal  trajectories,  it  follows  that  2(t,  T^  -  t  4  t^)  is  less  at  t  (t1  +  f)  for  the  non-optimal 

than  for  the  optimal  trajectory.  Taking  the  time  derivative  £’(t,  T)  of  £(t,  T)  in  Eq.  (76)  and 

evaluating  for  the  interval  t^  ^  t  ^  t^  yields 

2*(t,  T)  0  ,  •  (90) 

Since  2(t1#T^)  0,  by  Eq.  (90)  it  follows  that 

£(t,  T)  0  ,  t1  ^  t  ^  t2  (91) 

Since  Relation  (91)  holds  along  the  phase  u  +1  to  equilibrium  for  the  optimal  case,  it  follows 
that  2(t,  T^  —  t  4  t|)  <  0  at  t  (t^  4  c)  on  the  non-optimal  trajectory.  If  we  use  control  Law  (78) 

with  the  control  function  £(t,  T^  t  4  t^),  it  follows  that  we  switch  to  u  4l  at  t  (t^  4  c).  For 

the  interval  (t^  4  c)  ^  t  t^,  the  sign  of  £(t,  T^  —  t  4  t^)  remains  unchanged  for  both  the  optimal 

and  non-optimal  trajectories  except  that  x^  and  x^  are  less  for  the  non-optimal  than  for  the  op¬ 

timal  trajectory.  For  both  trajectories,  we  have  u  4l  over  the  time  interval  (t ^  4  c,  t  4  T^), 
but  x^  and  x^  lor  the  non-optimal  trajectory  are  both  less  than  for  the  optimal  until  x^  0  on  the 
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optimal  trajectory;  then,  2(t,  —  t  +  t^)  <  0  for  the  non-optimal  trajectory  until  x^  0.  Thus, 

for  the  non-optimal  trajectory,  the  sign  of  u  is  opposite  that  of  2(t,  —  t  +  t^)  over  the  inter¬ 

val  (t1  +  O  <  t<  (t1  +  T^. 

On  the  non-optimal  trajectory,  (x^x^)  ^  (0,  0)  at  t  =  (t^  +  T^);  in  fact,  x^  <  0  at  this  t, 

from  which  x.  becomes  zero  before  t  =  (t ,  +  T . ).  Let  the  time  t  at  which  x .  becomes  zero  be 
1  'll  1 

denoted  by  t  =  (t^  +  T^  —  A),  where  A  >  0.  When  the  point  at  t  =  (t^  +  —  A )  on  the  non- 

optimal  trajectory  described  above  is  attained,  one  may  start  over  again  with  a  function  2(t,T) 
for  arbitrary  T  and  repeat  the  argument.  This  process  can  be  continued  so  that  the  control 
Law  (78)  always  applies  for  appropriate  choices  of  £(t,T).  As  e  goes  to  zero,  the  non-optimal 
trajectory  approaches  the  optimal.  We  have  proved  the  following  theorem. 

Theorem  2. 

For  the  system  and  disturbance  of  Theorem  1,  optimal  control  is  attained 
in  practice  by  letting 

u  =  — sgn2(t,  T)  [Law  (78)] 

where  T  in  2(t,  T)  is  arbitrary  until  for  a  pair  of  values  (t^,  T^)  of  (t,  T) 

KV^)  =  o  . 

Thereafter,  control  Law  (78)  holds  with  T  -  (T^  —  t  +  t^)  until  x^  0. 

At  this  point,  a  new  function  2(t,T)  is  used  with  arbitrary  T  and  the 
process  is  repeated  so  that  control  Law  (78)  always  applies  for  appro¬ 
priate  choices  of  T  in  Z(t,T).  Thus,  the  single  control  Law  (78) 
suffices  to  yield  optimal  trajectories  to  equilibrium. 

VII.  EQUIVALENCE  OF  REFERENCE  INPUT  TO  LOAD  CHANGES 

In  the  preceding  theorems,  we  considered  the  load  disturbance  to  be  the  only  input  to  the 
system.  Now,  wc  shall  consider  the  system  of  Eqs.  (15)  subjected  to  two  inputs:  namely,  a 
reference  input  r(t),  and  the  load  disturbance  L(t).  We  shall  show  that,  in  effect,  the  addition 
of  a  reference  input  is  equivalent  to  replacing  the  disturbance  L(t)  in  Eqs.  (15)  by  a  new  disturb¬ 
ance  Lr(t)  with  no  reference  input.  Thus,  the  preceding  theorems  will  also  apply  to  the  system 
of  Eqs.  (15)  subjected  to  both  a  load  disturbance  L  and  a  reference  input  r. 

Let  e  be  the  difference  between  the  reference  value  r  of  the  controlled  variable  x^  given 


The  system  of  Eqs.  (15)  is  now  replaced  by 

e1  ’  =  u  -  L^  ,  |u|  1  (93) 

where 

Lr  =  L  +  r*  .  (94) 

It  follows  that  a  variation  in  the  reference  r(t)  is  equivalent  to  a  variation  r*  (t)  in  the  load  L^. 
The  variables  x^  and  x^  are  replaced  by  the  error  e  and  the  time  derivative  of  the  error  e’. 
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BEST  V 
LINEAR 

CONTROL  FUNCTION 


Fig.  5.  Experimental  results  af  system  response  ta  random  disturbance 
using  linear  and  nonlinear  control  functions. 


Fig.  6.  Phase  plane  trajectories  af  I  =  0  for  L^'  =  constant  =  a. 


18 


respectively.  Since  the  average  of  the  sum  of  two  variables  is  equal  to  the  sum  of  the  averages, 
we  may  write  the  average  of  L  and  L^  over  the  interval  of  time  T  as 

L  =  L  +  r' 
r 

IV,  IV  +  F'  .  (95) 


From  Eqs.  (76)  and  (95),  we  have  the  optimal  control  function  2(t,T)  for  the  equivalent  load 
given  by 


<(t,  T)  -  e(l  —  L^  sgne)^  +  y  e'  |  e 1 


1  +  2 


(H 


L  -  L  \ 

+  sgne' 


(96) 


For  L  to  be  controllable  by  Relation  (7),  we  must  have 


|L'  +  r"|  <  1  .  (97) 

We  have  shown  the  following  statement  to  be  true.  Let  the  system  of  Eqs.  (15)  be  in  equilibrium 
for  t  <  0  and  be  subjected  to  two  inputs  L(t)  and  r(t)  for  t  >  0,  where 

Lr(t)  -  L(t)  +  r'(t)  (98) 


and  L  is  made  up  of  an  initial  uncontrollable  portion  followed  by  a  controllable  portion  suffi¬ 
ciently  long  for  an  optimal  controller  to  bring  the  system  to  equilibrium.  According  to  Theorems 
1  and  2,  optimal  response  is  obtained  with  the  control  Law  (78)  and  the  control  function  given  in 
Eq.  (96). 

Consider  the  class  of  inputs  r(t)  which  may  be  closely  approximated  by  a  quadratic  function 
of  time,  i.e., 

r  3  ro  +  r4t  +  ry2  (99) 


for  constants  r  ,  r^,  and  r^.  Consider  also  the  class  of  disturbances  which  may  be  closely 
approximated  by  a  ramp  where 

L  s  L  +  at  (100) 

for  constant  a.  By  Relation  (94),  the  equivalent  disturbance  L^  is  a  ramp.  The  quantity  r  is 
an  arbitrary  value  for  t  <  0  and  there  may  be  a  step  in  L^  at  t  0.  By  Relations  (99)  and  (100), 
the  optimal  control  function  2(t,  T)  of  Eq.  (96)  may  be  approximated  closely  by 

£(t,  T)  =  e  |  1  —  L^  sgn  e|  +  -—  e *  |  e 1 1  .  (101) 

We  use  control  Law  (78)  with  2(t,T)  approximated  by  Relation  (101)  for  suboptimal  control. 
When  |  L^|  «  1,  the  optimal  control  function 

2  =  e  +  —  |  e 1 1  e'  (102) 


for  step  changes  is  effective  in  obtaining  suboptimal  response.  Experimental  results  were  ob¬ 
tained  using  the  control  function  of  Relation  (101)  with  the  control  Law  (78).  This  suboptimal 
control  was  effective  in  obtaining  suboptimal  response,  as  shown  in  Fig.  5. 

The  curves  2  0  for  2  of  Relation  (101)  for  various  values  of  L'  ,  where  IV  constant  a, 

are  shown  in  Fig.  6.  In  practice,  one  may  not  wish  or  be  able  to  measure  the  quantity  L^,  in 
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which  case  it  is  necessary  to  express  the  control  function  2  of  Relation  (101)  in  terms  of  e, 
e',  and  eM  only.  When  u  =  1,  Relations  (93)  yield 


Lr’  - 

I  r  I 


1  -  e* 


3 

sgn  L^  =  sgn(e"  -  eM  ) 
by  which  we  may  express  in  the  form 


(103) 


i  1  - 


sgn (eM  —  eM  ) 


(104) 


We  may  now  substitute  L^  from  Eq.  (104)  into  2  of  Eq.  (101)  and  obtain  a  control  function  in 
terms  of  e,  e',  and  eM  only.  Because  of  ever-present  noise,  some  filtering  of  eM  is  necessary 
in  order  to  employ  2  of  Eq.  (101)  with  of  Eq.  (104).  Normally,  the  system  of  Relations  (93) 
will  not  describe  the  system  exactly.  If  the  approximation  is  good,  the  control  will  be  suboptimal. 

Laboratory  tests  of  the  use  of  the  control  function  2  of  Eq.  (101)  gave  substantial  improve¬ 
ment  for  random  disturbances  over  what  could  be  obtained  by  known  techniques.  Here,  L^  is 
now  allowed  to  be  a  variable.  To  explain  the  remarkable  improvement  obtained  in  the  labora¬ 
tory,  we  considered  optimal  control  for  more  general  disturbances  than  ramps. 


Vm.  EXPERIMENTAL  RESULTS 

We  now  discuss  tests  which  were  run  to  evaluate  system  performance  for  random  disturb¬ 
ances  using  the  control  Law  (78)  with  the  control  function  of  Relation  (101)  which  is  optimal  for 
ramp  disturbances.  Thus,  if  the  disturbance  may  be  closely  approximated  with  ramps,  the 
control  function  of  Relation  (101)  will  be  a  good  approximation  to  the  optimal  control  function  of 
Eq.  (96)  and  will  yield  suboptimal  response. 

The  system  of  Eqs.  (15)  was  simulated  on  an  analog  computer  in  the  control  systems  labora¬ 
tory.  The  disturbance  L^(t)  was  generated  with  a  random  signal  generator  and  filtered  twice 
to  yield  controllable  and  uncontrollable  intervals  similar  to  those  shown  in  Fig.  1.  Control  Law 
(78)  with  the  control  function  of  Relation  (101)  were  simulated  along  with  the  system  to  yield 
closed-loop  control.  To  evaluate  system  performance,  a  second  system  was  simulated  identical 
to  the  first  except  for  a  linear  control  function  in  place  of  2  of  Eq.  (101).  The  responses  of  the 
two  systems  were  compared  over  long  duration  runs,  and  a  typical  section  of  these  runs  is  shown 
in  Fig.  5.  The  linear  control  function  used  in  the  second  system  was  of  the  form 

2  =  e  —eve1  +  /3eM  (105) 

for  constants  a  and  /3.  If  these  constants  are  properly  chosen,  the  linear  function  of  Eq.  (105) 
will  give  reasonably  close  response  to  that  obtainable  with  2  of  Eq.  (101)  for  a  given  disturb¬ 
ance.  See  Fig.  5  where  the  coefficients  in  Eq.  (105)  were  chosen  to  yield  a  minimum  rms  value 
of  the  controlled  variable  e  for  the  random  disturbance  L^  shown.  Many  of  the  same  type  tests 
shown  in  Fig.  5  were  run  and,  in  all  cases  where  the  disturbance  was  composed  of  short  uncon¬ 
trollable  sections  followed  by  long  controllable  sections,  the  control  function  of  Relation  (101) 
yielded  better  response  than  the  linear  function  of  Eq.  (105),  i.  e.,  shorter  error  response  dura¬ 
tion,  smaller  maximum  error,  and  fewer  overshoots.  It  was  also  determined  by  tests  with  2 
of  Eq.  (101)  with  the  L^  term  omitted  that  not  nearly  as  good  results  can  be  expected  for  this 
case.  Thus,  a  curve  L^(t)  is  better  approximated  by  a  broken  line  formed  by  secants  than  by 
using  a  staircase  approximation. 
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IX.  SUMMARY  AND  CONCLUSIONS 


The  problem  of  time  optimal  eontrol  of  a  seeond-order  system  initially  at  equilibrium  and 
subjected  to  a  elass  of  eommon  random  disturbances  is  solved.  The  class  of  disturbances 
treated  are  those  most  eommonly  encountered  in  praetiee,  and  are  made  up  of  an  initial  uncon¬ 
trollable  portion  followed  by  a  controllable  portion  of  sufficient  duration  to  enable  an  optimal 
controller  to  bring  the  system  to  equilibrium.  Necessary  conditions  for  optimal  eontrol  were 
derived  using  the  maximum  principle,  and  these  conditions  were  shown  to  be  satisfied  with  the 
eontrol  of  this  report.  Necessary  conditions  for  the  existence  of  optimal  trajectories  to  equilib¬ 
rium  were  also  derived.  These  are  restrictive  conditions  on  the  initial  state  of  the  system  and 
disturbance,  and  were  also  shown  to  be  satisfied  for  the  elass  of  disturbances  and  initial  states 
treated  here. 

A  eontrol  law  and  eontrol  function  were  derived  to  yield  optimal  trajectories  to  equilibrium, 
and  these  trajectories  were  shown  to  be  also  optimal  in  other  engineering  senses  sueh  as  that 
the  maximum  error  was  minimized  and  there  was  no  overshoot.  It  was  shown  that  a  single  eon¬ 
trol  law  with  a  single  eontrol  function  sufficed  to  yield  optimal  trajectories  to  equilibrium. 

Laboratory  tests  were  made  comparing  an  approximate  optimal  eontrol  function  with  the 
best  linear  eontrol  function.  These  tests  were  run  with  the  system  subjected  to  a  random  dis¬ 
turbance,  and  the  results  indicated  superior  eontrol  over  what  eould  be  obtained  by  known 
techniques. 
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13.  ABSTRACT 

This  report  concerns  the  time  optimal  control  of  a  system  variable  where  the  controlling  input  to  the  system  is  bounded,  as  is  normally 
the  case  in  practice.  Optimal  control  is  defined  here  as  that  control  which  yields  time  optimal  trajectories.  It  is  shown  that  time  optimal 
control  also  yields  optimal  trajectories  in  the  sense  of  minimizing  the  maximum  error  (if  this  is  the  initial  error,  minimize  the  overswing 
next)  and  the  number  of  oscillations.  The  problem  of  optimal  control  of  a  second-order  system  initially  in  equilibrium  and  subjected  to  a 
large  class  of  commonly  occurring  random  disturbances  is  solved  Disturbances  are  considered  to  be  controllable  or  uncontrollable.  The 
broad  class  of  random  disturbances  treated  herein  may  have  initial  nonequilibrium  values  and  consist  of  a  unidirectional  uncontrollahle 
portion,  followed  by  a  controllable  portion  of  sufficient  duration  to  enable  an  optimal  controller  to  bring  the  system  to  equilibrium.  A 
single  control  function  is  derived  which  suffices  to  yield  optimal  trajectories 

Previous  attempts  which  were  made  to  solve  similar  problems  using  a  statistical  approach  succeeded  only  in  obtaining  approximate 
optimal  control  for  disturbances  restricted  to  a  specific  class  such  as  white  noise.  Brownian  motion,  etc.  In  this  report,  the  optimal  con¬ 
trol  applies  to  a  much  larger  class  of  random  disturbances  than  previous  results,  in  that  it  does  not  restrict  the  disturbance  to  be  a  mem¬ 
ber  of  a  single  statistically  defined  class  such  as  white  noise,  etc.  The  disturbances  treated  here  contain  the  entire  class  of  disturbances 
most  commonly  occurring  in  practice.  This  is  true  because  in  modern  control  design  the  maximum  magnitude  of  the  controlling  input  is 
sized  such  that  most  of  the  time  the  disturbances  are  considered  controllable.  Thus,  the  most  general  class  of  commonly  occurring  dis¬ 
turbances  of  interest  to  the  control  designer  are  those  which  are  uncontrollable  for  a  short  duration,  followed  by  long  controllable  portions 

The  success  of  the  technique  used  in  this  report  lies  in  assuming  that  the  disturbance  is  initially  known  in  advance.  In  this  way.  a 
single  control  function  is  derived  that  will  yield  optimal  trajectories  for  as  large  a  class  of  disturbances  as  mathematically  possible 
This  is  the  best  that  can  theoretically  be  achieved.  Terms  contained  in  the  derived  control  function  yield  those  statistical  parameters 
which  are  actually  required  to  ohtain  optimal  control.  Once  the  basic  form  of  the  necessary  statistical  parameters  has  been  obtained,  the 
designer  can  estimate  or  measure  what  these  parameters  are  in  the  actual  system  for  practical  optimal  control.  Experimental  results 
verifying  this  fact  are  presented  here. 
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